Add normalize parameter to GATConv and GATv2Conv.

Part of #9831 for our final project for the Stanford CS224W course, this allows "GAT with Symmetric Normalized Adjacency Matrix" as described in “Bag of Tricks for Node Classification with Graph Neural Networks”.

Details

• Implementation of gat_norm inspired from gcn_norm, when edge_index is a SparseTensor, is_torch_sparse_tensor or dense torch Tensor.
• gat_norm is called after computing the alpha coefficients and return the updated values of edge_index and alpha. The outputs of gat_norm are passed as inputs of self.propagate.
• Update the docstring of GATConv and GATv2Conv.
• Add unit test cases.
• Override the add_self_loops parameter. We remove self loops from the initial graph before calling to gat_norm and add self loops with normalization in gat_norm as described in the paper. We tried to use the tools already provided in the library such as torch_sparse.fill_diag, to_edge_index, add_remaining_self_loops, add_self_loops and to_torch_csr_tensor.
• One concern is that there is no learned weight regardless of add_self_loops, because we explicitly remove self loops before edge update. This is consistent with the paper's description and gcn_norm, but different from the paper's implementation. Also, it seems that they use both out-degree and in-degree. We would appreciate your feedback on the preferred approach.
• When is_torch_sparse_tensor(edge_index) == True, we have an issue formatting back the index edge_index and the corresponding values in att_mat to the appropriate format. Our workaround consists of sorting lexicographically the values of att_mat, so it matches the index of edge_index for the propagate and update subsequent steps.
• When isinstance(edge_index, SparseTensor) and in the case we have multiple heads, e.g. num_heads > 1, we need to perform the operation $D \alpha$, which is a multiplication of SparseTensor (with values dimension greater than 1) and degree matrix. Our solution is based on repeating the degree matrix num_heads times. We don't use repeat_interleave directly as we encounter the following error: "repeat_interleave_cpu" not implemented for 'Float', but the current implementation should follow the same behavior.
• Only support non-bipartite graph mesasge passing.

Benchmarks

I have the following metrics with one T4 GPU, so it performs better for CiteSeer and PubMed dataset with a computation time cost.

with the following run commands:

python gat.py --dataset=Cora
python gat.py --dataset=Cora --normalize

python gat.py --dataset=CiteSeer
python gat.py --dataset=CiteSeer --normalize  

python gat.py --dataset=PubMed --lr=0.01 --output_heads=8 --weight_decay=0.001
python gat.py --dataset=PubMed --lr=0.01 --output_heads=8 --weight_decay=0.001 --normalize